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BACKGROUND OF THE INVENTION 

1. Field of the Invention : 

The present invention relates to medical imaging, and more particularly 
to extracting the myocardium from four-dimensional MR data (two- 
10 dimensional images over time and space). 


2. Discussion of Prior Art : 

Cardiovascular disease is the leading cause of death in the United 
States. Mortality has been declining over the years as lifestyle has changed, 

15 but the decline is also due to the development of new technologies to 

diagnose disease. One of these techniques is magnetic resonance imaging 
(MRI), which provides time-varying three-dimensional imagery of the heart. To 
help in the diagnosis of disease, physicians are interested in identifying heart 
chambers, the endocardium and epicardium, and measuring changes in 

20 ventricular blood volume (ejection fraction) and wall thickening properties over 
a cardiac cycle. The left ventricle is of particular interest since it pumps 
oxygenated blood out to distant tissue in the entire body. 

There has been a large amount of research on the analysis of medical 
images. Segmentation of these images has been particularly challenging. In 

25 the early nineties, researchers realized that tracking the cardiac wall motion in 
MR images could be used to characterize meaningful functional changes. A 
system proposed by S. R. Fleagle, D. R. Thedens, J. C. Ehrhardt, T. D. 
Schoiz, and D. J. Skorton, "Automated identification of left ventricular borders 
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from spin-echo resonance images", Investigative Radiology, 26:295-303, 
1991 , delineates the border of the myocardium using a minimum cost path 
graph search method after a user indicates the center of the left ventricular 
cavity and an area of interest, for example, with a mouse. D. Geiger, A. 
5 Gupta, L. A. Costa, and J. Vlontzos, "Dynamic programming for detecting, 
tracking, and matching deformable contours", IEEE Trans. PAMI, 17(3):294- 
302, 1995, used a dynamic programming approach to refine the contours 
specified by the user. A. Goshtasby and D. A. Turner, "Segmentation of 
cardiac cine MR images for extraction of right and left ventricular chambers", 

10 IEEE Trans. Medical Imaging, 14(1 ):56-64, 1 995, proposed a two step 

method combining intensity thresholding to recover blood from an image and 
a local gradient to outline strong edges using elastic curves. J. Weng, A. 
Singh, and M. Y. Chiu, "Learning-based ventricle detection from cardiac MR 
and CT images", IEEE Trans. Medical Imaging, 16(4):378-391, 1997, applied 

15 a threshold to an image based on parameters estimated during a learning 
phase to approximate the segmentation. 

However, no known system or method exists for providing an adaptive 
technique of analyzing cardiac images. Therefore, a need exists for a method 
of cardiac segmentation combining edge, region and shape information in a 

20 deformable template. 

SUMMARY OF THE INVENTION 

According to an embodiment of the present invention, a method is 
provided for segmenting a magnetic resonance image of interest of a left 
25 ventricle. The method includes determining a myocardium contour according 
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to a graph cut of candidate endocardium contours, and a spline fitting to 
candidate epicardium contours in the absence of shape propagation. The 
method further includes applying a plurality of shape constraints to candidate 
endocardium contours and candidate epicardium contours to determine the 
5 myocardium contour, wherein a template is determined by shape propagation 
of a plurality of magnetic resonance images in a sequence including the 
magnetic resonance image of interest 

According to an embodiment of the present invention, a method for 
segmenting a magnetic resonance image of interest of a left ventricle includes 

10 determining a myocardium response image according to a histogram of pixel 
intensity in magnetic resonance image of interest. The method includes 
determining a plurality of candidate contours, according to a plurality of 
energy functions, to which a plurality of confidence values are assigned. The 
method further includes applying a plurality of shape constraints to the 

15 candidate endocardium contours and the candidate epicardium contours to 
. determine a myocardium contour. 

The myocardium contour is based on a plurality of magnetic resonance 
images in a sequence including the magnetic resonance image of interest, 
wherein the myocardium contour includes an endocardium contour and an 

20 epicardium contour. 

The method determines an approximate contour pair of the left 
ventricle according to a plurality of points defined in a gray level profile of the 
image as an intersection of a plurality of cross sections of the left ventricle. 
The approximate contour pair is determined according to a Hough transform 

25 array to vote for a center position and a radius of a myocardium centerline. 
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Determining the myocardium response image according to the 
histogram of pixel intensity further includes fitting a mixture of Gaussians to 
the histogram of pixel intensity to determine portions of blood, muscle and air 
in the image. 

5 For each candidate closed contour, the method further includes 

defining a search space around an approximate contour, defining a line of 
source nodes and sink nodes, and determining a shortest path between the 
source nodes and the sink nodes. The method includes defining a new source 
node and a plurality of new sink nodes, and determining a shortest path 

10 between the new source node and new sink nodes. 

The candidate endocardium contours and the candidate epicardium 
contours are each determined according to a combination of a gradient 
magnitude and a direction using a cross product of a gradient direction and a 
contour direction. The method further includes determining a first candidate 

15 endocardium contour according to a positive cross product of a gradient 

direction in the input image and a contour direction, and determining a second 
candidate endocardium contour according to a negative cross product of a 
gradient direction in the myocardium image and a contour direction. The 
method includes determining a first candidate epicardium contour according 

20 to a gradient magnitude in the input image, and determining a second 
candidate epicardium contour according to a positive cross product of a 
gradient direction in the myocardium image and a contour direction. 

The method determines a one-to-one match matrix, wherein points on 
the endocardium contour and epicardium contour are assigned to a plurality of 
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template points, wherein the template points are determined from propagation 
through a plurality of magnetic resonance images. 

According to an embodiment of the present invention, a method is 
provided for segmenting a magnetic resonance image of interest of a left 
5 ventricle. The method includes determining a myocardium response image 
according to a histogram of pixel intensity in magnetic resonance image of 
interest, determining a plurality of candidate contours, according to a plurality 
of energy functions, to which a plurality of confidence values are assigned. 
The method further includes determining a myocardium contour according to 

10 a graph cut of candidate endocardium contours, and a spline fitting to 
candidate epicardium contours in the absence of shape propagation. The 
method applies a plurality of shape constraints to the candidate endocardium 
contours and the candidate epicardium contours, to determine the 
myocardium contour, wherein a template is determined by shape propagation 

15 of a plurality of magnetic resonance images in a sequence including the 
magnetic resonance image of interest. 

The myocardium contour includes an endocardium contour and an 
epicardium contour. 

The method determines an approximate contour pair of the left 

20 ventricle according to a plurality of points defined in a gray level profile of the 
image as an intersection of a plurality of cross sections of the left ventricle, 
wherein the contour pair comprises an endocardium contour and an 
epicardium contour. The approximate contour pair is determined according to 
a Hough transform array to vote for a center position and a radius of a 

25 myocardium centerline. 
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Determining the myocardium response image according to the 
histogram of pixel intensity further comprises fitting a mixture of Gaussians to 
the histogram of pixel intensity to determine portions of blood, muscle and air 
in the image. 

For each candidate closed contour, the method further comprises 
defining a search space around the myocardium response image, defining a 
line of source nodes and sink nodes, and determining a shortest path 
between the source nodes and the sink nodes. The method includes defining 
a new source node and a plurality of new sink nodes, and determining a 
shortest path between the new source node and new sink nodes. 

The candidate endocardium contours and the candidate epicardium 
contours are each determined according to a combination of a gradient 
magnitude and a direction using a cross product of a gradient direction and a 
contour direction. The method further comprises determining a first 
candidate endocardium contour according to a positive cross product of a 
gradient direction in the input image and a contour direction, and determining 
a second candidate endocardium contour according to a negative cross 
product of a gradient direction in the myocardium image and a contour 
direction. The method includes determining a first candidate epicardium 
contour according to a gradient magnitude in the input image, and 
determining a second candidate epicardium contour according to a positive 
cross product of a gradient direction in the myocardium image and a contour 
direction. 

The graph cut further includes defining a graph wherein each node 
corresponds to a connected component region between candidate pixels on 


the candidate contours, and determining a minimum cut between a center 
node and an outside node. 

The spline fitting further includes fitting a spline through points of the 
candidate contours. 

According to an embodiment of the present invention, a program 
storage device is provided readable by machine, tangibly embodying a 
program of instructions executable by the machine to perform method steps 
for segmenting a magnetic resonance image of interest of a left ventricle. The 
method includes determining a myocardium response image according to a 
histogram of pixel intensity in magnetic resonance image of interest, 
determining a plurality of candidate contours, according to a plurality of 
energy functions, to which a plurality of confidence values are assigned. The 
method further includes determining a myocardium contour according to a 
graph cut of candidate endocardium contours, and a spline fitting to candidate 
epicardium contours in the absence of shape propagation. The method 
applies a plurality of shape constraints to the candidate endocardium contours 
and the candidate epicardium contours, to determine the myocardium 
contour, wherein a template is determined by shape propagation of a plurality 
of magnetic resonance images in a sequence including the magnetic 
resonance image of interest. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Preferred embodiments of the present invention will be described 
below in more detail, with reference to the accompanying drawings: 


Fig. 1 is a flow chart showing a method of segmentation according to 
an embodiment of the present invention; 

Fig. 2a shows an input image according to an embodiment of the 
present invention; 

Fig. 2b shows a histogram of the input image of Fig. 2a, fitted with a 
mixture of Gaussians according to an embodiment of the present invention; 

Fig. 2c shows a myocardium response image according to an 
embodiment of the present invention; 

Figs. 3a-b show an example of two passes of Dijkstra's method 
according to an embodiment of the present invention; 

Figs. 4a-f show an example of applying shape constraints to recover 
the myocardium contours according to an embodiment of the present 
invention; 

Figs. 5a-d show an example of a graph cut method according to an 
embodiment of the present invention; and 

Figs. 6a-b are graphs showing a cumulative distribution of error 
distances between true contours and segmented contours, over all points, 
over all images, and over all patients. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

The present invention can be embodied in any suitable commercial 
cardiac analysis package, such as the ARGUS cardiac analysis package from 
Siemens, which offers a complete system of drawing tools and automatic 
segmentation methods to allow the physician to outline the myocardium in 
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each image in the patient data set, determine volumes, ejection fraction, and 
perform a thickening analysis. 

Referring to Fig. 1, illustrating a method according to an embodiment of 
the present invention, the method determines approximate contours of the left 
ventricle 102 and a myocardium response image 104 according to a 
histogram of pixel intensity in the magnetic resonance image of interest. The 
method determines a plurality of candidate closed contours of the left ventricle 
106, according to a plurality of energy functions. Further, the method is aware 
of whether shape information exists for the magnetic resonance image of 
interest 108 and is able to apply a desired method to determine a myocardium 
image. For example, in the presence of shape information, the method 
determines a template from propagation through a plurality of magnetic 
resonance images, and determines an endocardium and epicardium 
according to a matching technique between the candidate contours and the 
template. The matching technique can include, for example, point-pattern 
matching and shape matching. The method determines an endocardium 
contour according to a graph cut 1 12 and an epicardium contour according to 
a spline fitting 1 14 to the candidate contours for a singular image, where a 
template is not present. 

It is to be understood that the present invention may be implemented in 
various forms of hardware, software, firmware, special purpose processors, or 
a combination thereof. In one embodiment, the present invention may be 
implemented in software as an application program tangibly embodied on a 
program storage device. The application program may be uploaded to, and 
executed by, a machine comprising any suitable architecture. Preferably, the 


machine is implemented on a computer platform having hardware such as 
one or more central processing units (CPU), a random access memory 
(RAM), and input/output (I/O) interface(s). The computer platform also 
includes an operating system and micro instruction code. The various 
processes and functions described herein may either be part of the micro 
instruction code or part of the application program (or a combination thereof) 
which is executed via the operating system. In addition, various other 
peripheral devices may be connected to the computer platform such as an 
additional data storage device and a printing device. 

It is to be further understood that, because some of the constituent 
system components and method steps depicted in the accompanying figures 
may be implemented in software, the actual connections between the system 
components (or the process steps) may differ depending upon the manner in 
which the present invention is programmed. Given the teachings of the 
present invention provided herein, one of ordinary skill in the related art will be 
able to contemplate these and similar implementations or configurations of the 
present invention. 

According to an embodiment of the present invention, a method is 
provided which automatically hypothesizes two concentric circles at the 
location of the left ventricle in an MR image. However, due to the relative 
symmetry of the left ventricle and computational constraints, the method uses 
the gray values of the pixels along four cross sections through the ventricle 
instead of the entire region to determine a feature set. A method of automatic 
localization is described in Attorney docket no. 99P9036US01, entitled, 
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Method for Learning-Based Object Detection in Cardiac Magnetic Resonance 
Images, and is incorporated by reference herein in its entirety. 

The present invention patterns a Markov process. A Markov analysis 
looks at a sequence of events, and analyzes a tendency of an event to be 
followed by another event, generating a new sequence of random but related 
events. According to an embodiment of the present invention, in a learning 
phase, positive and negative examples are presented and the method finds 
the ordering of the Markov process that maximizes a separation (minimizes 
the Kullbach distance, also known as the relative entropy or cross entropy) 
between the two classes in the training set. The method includes a detection 
phase, during which a test image is scanned and each location is assigned to 
a class based on the log-likelihood ratio. 

Neighboring positions classified as the left ventricle are portioned into 
clusters. The method defines eight salient points in the gray level profiles as 
the intersection of the four cross sections with the ventricle's medical axis. 
Average profiles are built from training example profiles aligned using a curve 
registration technique, for example, as proposed by Ramsay and Xi. The 
cross sections of each of the cluster candidates are warped onto 
corresponding average profiles. The location of the salient points in the image 
are then accumulated using a Hough transform array to vote for the most 
likely center position and radius for the myocardium centerline. 

Typically, the user segments one image, for example, the slice closest 
to the valves at end-diastole (ED base) and propagates the segmented 
contours to all the slices in the ED phase (ED propagation). The ED contours 
can be propagated to the end-systole (ES) phase (ES propagation) to 
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compute the ejection fraction. For a more detailed analysis, the user can also 
propagate all the ED contours to all the images in all the phases using 
temporal propagation. 

For ED propagation, the present invention uses the automatic 
localization method described above. However, since the ventricle size is 
approximately known from the template image, the scale search can be 
limited, for example, to between 0.85 and 1.15 times the size of the template. 
The method can limit the search space for the location of the ventricle, for 
example, to 30 pixels around the location of the template. For ES propagation, 
the method knows the location of the left ventricle has not changed and 
therefore, the template contours can be scaled. The endocardium is scaled by 
0.6 and the epicardium by 0.9. In the case of temporal propagation, both the 
location and scale of the contours do not change significantly. Therefore, the 
method can copy the contours from one image to the next. 

In MR imaging, the intensity of a pixel depends on the properties of the 
tissue being imaged. As shown in Fig. 2a, in an MR image of the left ventricle, 
the blood is bright 201 , the muscles are somewhat dark 202, but not as dark 
as the air-filed lungs 203. This fact can be verified by looking at the histogram 
of a region around the myocardium as shown in Fig. 2b. The method uses an 
Expectation-Maximization (EM) method to fit a mixture of three Gaussians to 
the histogram. The method then creates a myocardium response image, 
showing the probability that a pixel belongs to the middle Gaussian which 
corresponds to the myocardium. It can be seen from Fig. 2c that the left 
ventricle myocardium 204 is highlighted, but neighboring organs, e.g., 205, 
are also highlighted. 
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To complement the results of region segmentation, the method uses an 
active contour formulation similar to Geiger's dynamic programming approach 
or Mortensen and Barrett's Dijkstra's approach. The advantage of these graph 
theoretic methods over a gradient descent approach as proposed by Kass et 
al. is that they are able to recover the global optimum of the energy function 
and are therefore insensitive to the initial contour position. 

Geiger defines the energy of a contour (pi... p n )as: 

^'••■'' )= fl^^ + # /fe) -^M < 1 > 

where ||v/(p| is the magnitude and V/(/>) is the direction of the image 
gradient at pixel p. This is equivalent to finding a shortest path in a graph 
where nodes correspond to pixels and the cost of a link between two 
neighboring pixels is defined as: 

Given an approximate contour in an image, the method places a 
symmetrical search space 304 around it and defines a line of source nodes 
306, connected to a "pseudo" source node, and sink nodes 308. Dijkstra's 
method then finds the shortest path between the pseudo source node and 
one of the sink nodes 302. The method defines a new single source point 312 
in the middle of the recovered contour and a set of sink nodes 310 
neighboring the source node, and does a second pass of Dijkstra's method to 
produce a closed contour 314. 

To combine the information provided by both the image and the 
myocardium response image, Dijkstra's method is implemented with two 
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different energy functions. Each run gives different candidate points for the 
contours, along with a confidence based on the contribution of each point to 
the total energy function. One energy function combines gradient magnitude 
and direction using the cross product of the gradient direction and the contour 
direction. In this case, the contour is built clockwise by Dijkstra process and 
the image gradient points from bright to dark. To separate a bright region 
inside from a dark region outside, resp. a dark region inside from a bright 
region outside, the z component of the cross product between the image 
gradient and the contour direction should be positive, resp. negative. The 
energy is set to a large number otherwise. The cost of a link between two 
pixels is: 


e(l,z>0, Pl ,p 2 ) = 


I fa -*i)sin(v/(p 2 ))- 

j— Z if z= (y 2 -^)cos(v/(p 2 ))>0 (3) 

r^U^MI +£ otherwise 

where e is a small constant, for example, 0.001, to bound the energy function. 
E(/,z>0) and E(H,z<0) can be used for the endocardium, where / is the input 
image and H is the myocardium response image. For epicardium, E(H,z>0) 
can be used. Since the gradient direction in / outside the myocardium flips 
between the bright right ventricle and the dark lungs, E'(l) can be used, which 
can be defined as: 

Different energy functions highlight different features of the 
myocardium. 
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Dijktra's method is applied with various energy functions and all points 
on the resulting contours are candidate points for the final contour. When the 
contours are propagated from one image to the next, either spatially or 
temporally, it is reasonable to assume that the shape of the contours does not 
change drastically. Based on this assumption, the method can determine 
which parts of which contours are correct. 

A shape alignment method, for example, as proposed by Duta et al., is 
used to establish a correspondence between a subset A' of the template 
points >4={v4 y }j =1i ... a and a subset B' of the candidate test points B={B*} k =i b . 
According to an embodiment of the present invention, both contours are 
considered as one shape. Given a pair of "corresponding" points in A and B, a 
rigid similarity transform is hypothesized to align the contours. A one-to-one 
match matrix M is determined to assign every point B to its closest neighbor in 
A if the distance is less than a threshold. This allows the following equation: 


A m ) = -^1L w jI x a j +cy Bj ~b) +{y Ai -ay B] +cx Bj -d) 


+ - (5) 
n 


where n is the number of correspondences and (a,b,c,d) are the parameters 
of the similarity transform. The weight wj is set to be the confidence of the test 
point B ; -. The goal is to minimize f{M). It is not possible to evaluate all possible 
quadruplets of points, so the 10% of the points with the largest confidence in 
the test set are chosen and paired with points from the same contour in the 
template set. One of ordinary skill in the art would recognize in light of the 
present invention that larger and smaller sets of points can be chosen. 

Fig. 4a shows the input image and Fig. 4b illustrates four recovered 
contours, 401-404, where darker points show higher confidence. Fig. 4c 
shows the template shape from the segmentation of the previous image. Fig. 
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4d shows the established correspondences for the best similarity transform. 
The shape constraints allow the method to choose the outside candidate 
points for the endocardium 407 rather than the inside points that outline the 
papillary muscles 406, and the inside candidate points for the epicardium 405 
rather than the outside points that outlined the fat 408. Once the 
correspondences are established, the template shape is warped by moving 
the template points to their corresponding test points as in Fig. 4e. The 
contours are smoothed using the method proposed by Xu et al., which 
minimizes shrinkage. A final segmentation result is shown in Fig. 4f. 

When an image needs to be segmented on its own, not in the context 
of propagation, there is no shape information available to the system. The 
endocardium should be pushed as far as possible away from the center so 
that it outlines the myocardium rather than the papillary muscles, and the 
epicardium should stay close the endocardium. To enforce these constraints, 
the method modifies the confidence values in the following way: 


where Q is the center of mass of the confidence points. 

For the endocardium, the method determines the cycle with a 
maximum confidence. Fig. 5a shows an example of a confidence image. A 
graph, for example, as illustrated in Fig. 5c, is defined where each node 
corresponds to connected component regions, as shown in Fig. 5b, between 
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confidence pixels on the candidate contours. The weight of an edge between 
two nodes is inversely proportional to the confidence of pixels on the common 
boundary. Then, the method determines a minimum cut between the center 
node and the outside node. A max flow method is used to determine the 
minimum cut. Fig. 5d shows an example of a final segmentation, or 
myocardium contour (shown in dark). 

For the epicardium, the method needs a smooth contour since there is 
no clear edge between the myocardium and the liver, and the right ventricle 
myocardium appears to merge into the left ventricle myocardium. Therefore, 
the method fits a spline to the points of the two candidate contours. Spline 
curves originate from flexible strips used to create smooth curves. Like Bezier 
curves, splines are formed from piecewise approximations of cubic polynomial 
functions with zero, first and second order continuity. B-Splines are an 
example of a particular type of spline. 

In order to test our algorithm, we have collected 29 patient data sets 
along with a manual segmentation of the ED and ES phases by radiologists, 
for a total of 458 segmented images. The images were acquired on Siemens 
MAGNETOM systems using two different pulse sequences. FLASH pulse 
sequences were traditionally used for MR cineangiography. Siemens recently 
pioneered the TrueFISP pulse sequences for cardiac cine imaging which 
present higher contrast-to-noise ratio without affecting temporal or spatial 
resolution. We collected 22 TrueFISP patients and 7 FLASH patients. Our 
database presents a great variety of heart shapes, image contrast, and edge 
crispness. The difficulty with FLASH images is that the edges can be blurred. 
The challenge with TrueFISP images is that the papillary muscles are so well 
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defined that it can be difficult to avoid outlining them. Further, the user may 
not want to outline them and the shape constraint can help in this task. 

The method automatically segments ED and ES phases of all the 
datasets. To compare the automatic contours A with the true contours 8, the 
5 method determines the distances d(a,B)=mm beB \\a-b\\ for all points a in the 
automatic contour. The method also determines d(b,A) for all points b in the 
true contour. Fig. 6 shows the cumulative distribution of these error distances 
over all ED and ES images of all 29 datasets. It can be seen that, on the 
average, the error is less than one pixel and an error of five pixels or more is 

10 made less than five percent of the time. 

The present invention proposes a method for segmenting the left 
ventricle in cardiac MR images. The method combines edge information, 
region information obtained from an EM fitting of a mixture of Gaussians to a 
histogram, and shape information through a point pattern matching strategy. 

15 The method can be integrated into, for example, the ARGUS cardiac analysis 
package from Siemens. 

Having described embodiments for cardiac segmentation combining 
edge, region and shape information in a deformable template, it is noted that 
modifications and variations can be made by persons skilled in the art in light 

20 of the above teachings. It is therefore to be understood that changes may be 
made in the particular embodiments of the invention disclosed which are 
within the scope and spirit of the invention as defined by the appended claims. 
Having thus described the invention with the details and particularity required 
by the patent laws, what is claimed and desired protected by Letters Patent is 

25 set forth in the appended claims. 
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